Video encoding with constrained fluctuations of quantizer scale

ABSTRACT

During video compression quantizer scale values for respective blocks under control of a complexity of a content of image information. The temporal and or spatial fluctuations of the quantizer scale values are dampened, for example by low pass filtering provisionally selected quantizer scale values. The quanitizer scale values are use to quantize the video data. The quantizer scale values may also be used during transcoding.

The invention relates to a method of encoding video data and a systemimplementing such a method.

An article titled “On performance gains in MPEG-2 video coding via arate-distortion optimal route” by W. M. J. Coene and G. J. Keesman,published in “Signal Processing” 55 (1996) pp. 369-374, disclosesmethods of computing MPEG encoded video data. The MPEG video encodingstandard provides for compression of video information. Video frames arecoded as I (Intra) frames that are encoded by themselves or P and Bframes that are encoded in terms of updates to other frames. Videoframes are divided into blocks (MPEG distinguishes blocks andmacroblocks, the latter comprising a number of blocks with informationfrom different color channels, but since this does not affect theinvention, the general term “block” will be used indiscriminately forblocks and macroblocks unless mentioned otherwise). Some blocks are madezero, a DCT (Discrete Cosine Transform) coefficients are computed of thevideo information in each remaining block, the coefficients arequantized and the quantized coefficients are encoded into an MPEGsignal.

Encoding by means of updates, zeroing blocks and quantizing are the maincontributors to compression of video information in MPEG. Quantizinginvolves replacing the coefficients with quantized coefficients selectedfrom a limited set of available values. The limited set is characterizedby a quantizer scale value Q, which corresponds to the differencebetween successive available values. Replacement of the coefficients byquantized coefficients reduces the amount of information that is neededto encode the coefficients, be it at the expense of introducingdistortion of the image by means of rounding errors due to thedifferences between the original coefficients and the quantizedcoefficients. In an MPEG signal the quantizer scale value Q may bevaried from block to block, so as to select an optimal balance betweencompression and distortion of the encoded signal.

The article by Coene and Keesman discloses an optimization method forselecting optimal compression parameters, including the quantizer scalevalues. A measure of distortion is minimized under the constraint thatno more than a certain amount of information (bit-rate) may be used toencode the video signal. The particular measure of distortion used inthis method is not disclosed, but presumably an aggregate of distortionsof individual frames is used.

The article shows that in prior methods the distortion of I frames onone hand differed the distortion of B and P frames on the other hand.Thus, the distortion varied periodically with the so-called group offrame frequency (a group of frames containing an I frame and surroundingB and P frames). In one embodiment of the method of the article reducesthis variation of distortion was reduced by optimizing an aggregate ofthe distortions of a plurality the frames in such a group. In anotherembodiment this variation was reduced by making the distortion of allframes equal.

However, in the known method of encoding, human viewers still noticevisual artifacts for some video signals when the encoded signal isdecoded. For instance, at low encoding bit-rates a heartbeat effect maybe observed due to alternating periods of better and worse coding ofblocks. This heartbeat effect is most noticeable in difficult scenes,for instance, during zooming in on or out of a particular object, or atedges of relatively uniform surfaces, which are liable to have beenencoded with fewer bits and are therefore prone to more errors. Thiseffect may also be present in transcoded data streams and is then due tothe buffer regulation algorithm.

It is an objective of the invention to reduce the above mentionedheartbeat effect.

The invention is based on the realization that the noticeable distortiondue to quantization is not so much a matter of distortion of frames,i.e. difference between compressed and uncompressed images, but rather amatter of local variation of the distortion. This effect is not measuredby normal aggregate measures of distortion.

By using the encoding method according to the invention, thefluctuations of the quantizer scale of blocks at the same position inthe frames as a function of time and/or in the same image as a functionof position are dampened. In an embodiment of the invention, a spatialand/or temporal filter is applied over the pre-selected quantizer scalevalues, thereby limiting a change in the quantization scale as afunction of time and/or position.

The invention aims to locally constrain fluctuations about a trendaround which the quantizer scale may fluctuate. The quantizer scale mayfollow the trend, but temporal or spatial fluctuations around the trendare constrained, so that the effect of fluctuations in a localenvironment in the temporal or space domain is damped. The inventionfurther relates to an encoder an audiovisual device, a data containerdevice, a computer program and a data carrier device on which a computerprogram is stored.

In an embodiment the method is applied to transcoding of a video streamthat has already been quantized. Thus generation of artifacts duringtranscoding is minimized.

In another embodiment estimated motion vectors are used to select blocksin different frames, that are related to each other by the motionvectors. The provisionally selected quantization values for the blocksthat found to successively correspond with each other in successiveframes are temporally low-pass filtered.

Further objects, elaborations, modifications, effects, and details ofthe invention appear from the following description, in which referenceis made to the drawing.

FIG. 1 shows an encoding apparatus;

FIG. 2 shows a flow diagram of an encoding method;

FIG. 3 shows a graph of quantizer scale as a function of time

FIG. 4 shows a transcoding apparatus.

FIG. 1 shows an encoding apparatus. The encoding apparatus has an inputand an output 17 and the apparatus contains a pre-processor 12, aquantizer 14, a packager 16, a quantizer scale selector 18 and aquantizer scale filter 19. The input is coupled to the output via,successively, pre-processor 12, quantizer 14 and packager 16. Quantizerscale selector 18 has an input coupled to input 10 and an output coupledto a quantizer scale input of quantizer 14 via quantizer scale filter19.

FIG. 2 will be used to explain operation of the apparatus in terms ofMPEG encoding. In a first step 21 during operation video inputinformation is supplied to input 10. In the first step 21 pre-processor12 pre-processes the video input information, splitting frames intoblocks, performing a DCT (Digital Cosine Transform) of video informationin each block. Pre-processor 12 supplies the resulting transformcoefficients to quantizer 14.

In a second step 22 quantizer scale selector 18 pre-selects quantizerscale values Qp, quantizer scale filter 19 filters these pre-selectedquantizer scale values Qp temporally and/or temporally and supplies thefiltered quantizer scale values Qf to quantizer 14.

In a third step 23 quantizer 14 quantizes the coefficients received frompre-processor 12, i.e. it replaces the received coefficients by nearbyquantized coefficients from a limited set of quantized values andsupplies the quantized coefficients to packager 16. This may be done forexample by dividing the coefficients of a block (optionally minus anoffset) by the quantizer scale value and taking the integer part of thequotient as quantized value. In a fourth step 24 packager 16 encodes andpackages the quantized coefficients into an MPEG compatible signal andsupplies that signal to output 17. The steps 21, 22, 23, 24 areperiodically repeated as new frames come in.

In second step 22 quantizer scale selector 18 pre-selects quantizerscale values Qp for respective blocks in respective frames of theincoming video information. Quantizer scale selector 18 supplies thepre-selected quantizer scale values Qp to quantizer 14 via quantizerscale filter 19 to control the step size between successive quantizedvalues in the limited set of quantized values from which quantizer 14selects the quantized coefficients.

Any known method of selecting the quantizer scale values may be used topre-select the quantizer scale values Qp. In one example quantizer scaleselector 18 receives a bit-rate selection signal and pre-selects thequantizer scale values Qp so that on average the bit rate of the outputsignal at output 17 does not exceed the selected bit rate. The quantizerscale selector 18 computes a measure of complexity for each block andallocates individual bit rates to each block in proportion to thecomplexity and so that the aggregate bit-rate of all blocks does notexceed the selected bit-rate. The quantizer scale value Qp of each blockis then selected so as no to exceed the bit-rate allocated to thatblock.

Quantizer scale selector 18 supplies the pre-selected quantizer scalevalues Qp to quantizer 14 via quantizer scale filter 19. Quantizer scalefilter 19 performs a temporal and/or spatial low pass filter operationon the pre-selected quantizer scale values Qp and supplies the low passfiltered quantizer scale values Qf to quantizer 14.

It will be appreciated that second step may involve a certain amount oflookahead. That is, pre-selected quantizer scale values may be computedfor blocks of a frame before the blocks of preceding frames and/orpreceding blocks in the same frame are quantized, so as to select thequantizer scale value Qf for those blocks of preceding frames and/orpreceding blocks in the same frame. Thus, account may be taken ofpre-selected quantizer scale values of future blocks.

FIG. 3 shows a graph 30 of pre-selected quantizer scale values Qfselected by quantizer scale selector 18 for blocks at the same pixelpositions in successive frames of video information as a function oftime (i.e. frame number). FIG. 3 also shows a graph 32 of temporallyfiltered quantizer scale values Qf, as produced by quantizer scalefilter 19. It will be noted that the filtered quantizer scale values Qf32 follow the general trend of the preselected quantizer scale valuesQp, but that fluctuations around the trend are suppressed.

Similar graphs may be drawn for the quantizer scales Qp and Qf as afunction of the position of the blocks with which these scales areassociated in a frame.

Any method may be used to filter the pre-selected quantizer scale valuesQp 30. For example one might take the median of a number of successivepre-selected quantizer scale values Qp in a window of frames around theframe for which the filtered quantizer scale value Qf is selected. Thisremoves fluctuations but allows sudden changes, for example when thecontent of the block changes abruptly. In another example filtering isrealized by computing an average of successive pre-selected quantizerscale values Qp in a window of frames around the frame for which thefiltered quantizer scale value Qf is selected. The average may be aweighted average, which assigns more weight to pre-selected quantizerscale values Qp from frames close to the frame for which the filteredquantizer scale value Qf is selected.

In another embodiment motion information is used to support filtering.In this case, filter 19 temporally filters the quantization values thatare selected for blocks in different frames that are related to eachother by motion vectors. That is, if it is estimated that the content ofa first block in a first frame moves to a second block in a second frameand from there to a third block in a third frame and so on, temporalfiltering is applied to the sequence of quantizer scale values formed bythe quantizer scale values of the first block, the second block, thethird block and so on and the filtered quantizer scale values are usedto quantize the corresponding blocks. Again this reduces visibleartifacts. The motion vectors may be obtained in any way, for examplefrom the motion vectors used in MPEG encoding.

It will be appreciated that, although the implementation shown in FIG.1, using a quantizer scale value filter 19 is advantageously simple,other means for reducing fluctuations may be used. For example, a searchmay be used for a set of quantizer scale values that minimizesdistortion without exceeding a predetermined bit-rate. In this case, theapparatus should use a modified measure of distortion that assigns anincreased value to the distortion in case of fluctuations in thequantizer scale values. This may be done, for example, by using ameasure of distortion based on the deviations between the original imageand the encoded image, added to squares of differences between actuallyused quantizer scale value Qf at corresponding positions in adjacentframes and/or between adjacent positions in the same frame.

It will also be appreciated that, although the principles of reducingfluctuations in quantizer scale have been described, many modificationsmay be needed to account for peculiar details of the coding standard.For example, in case of the MPEG standard only a limited number ofquantizer scale values are allowed. Therefore, the filtered quantizerscale values Qf should be selected from this limited number of values,but the preselected quantizer scale values Qp may be selected freely.Also the MPEG standard imposes some restrictions on differences betweenthe quantizer scale values that may be used for different blocks in amacroblock. This can be realized for example by selecting filteredquantizer scale values Qf per macroblock.

Furthermore, in the MPEG standard most changes in the quantizer scalevalue (e.g. changes between horizontally adjacent blocks) have to besignaled with additional signals, which are not needed when thequantizer scale does not change. The additional signals lead toincreases in the number of bits in the compressed signal. Therefore, itmay be desirable to weigh both the noticeable distortion due to changesof the quantizer scale and the increases in the number of bits in thecompressed signal.

The invention may be applied to compression that produces data fortransmission or for storage of video information. The invention can beapplied to data being stored to a data container device like a floppydisk a Digital Versatile Disc or a Super Audio CD, or a master orstamper for manufacturing DVDs or SACDs. The invention may also beapplied to transcoding of compressed video information.

FIG. 4 shows a transcoding apparatus. The transcoding apparatus theapparatus contains a partial decoding unit 40, a requantizer 44, apackager 46, a quantizer scale selector 48 and a quantizer scale filter49. The input of the transcoder is coupled to the output via,successively, partial decoding unit 40, requantizer 44 and packager 46.Quantizer scale selector 48 has an input coupled to an output of thepartial decoder and an output coupled to a quantizer scale input ofrequantizer 44 via quantizer scale filter 49.

In operation the transcoder receives compressed encoded data with signalvalues that have been quantized using a first quantizer scale. Thetranscoder requantizes the signal values with a second quantizer, forexample in order to increase the compression rate. Partial decoding unit40 partially decodes a received compressed signal, and suppliesinformation representing the quantized signal value from the receivedsignal to the requantizer 44. Requantizer 44 selects a new quantizedvalue for each signal value, so that the new quantized value belongs toa quantization scale indicated by filter 49. Packager 46 repackages theinformation from the stream into a new encoded stream, encoding the newquantized values.

Selector 48 selects quantization scales. This is done for example foreach block in proportion to the quantization scales used incorresponding block in the incoming signal (the proportionality factorbeing selected so that a measured bit rate at the output on averagecorresponds to an externally demanded bit rate). Filter 49 spatiallyand/or temporally filters the selected quantization scale values asdescribed for FIG. 1.

Although separate hardware units have been shown in an implementation ofthe invention, it will be appreciated that the functions of one or moreof these units may be combined in a single unit. For example,preselection and filtering, performed by units 18, 19, may be executedby a single unit. Furthermore, the components of the apparatus shown inFIG. 1 may be physically distributed over a number of apparatuses, whilelogically regarded as a single apparatus. Although the invention hasbeen described in terms of compression according to the existing MPEGstandard, it may of course be applied to any form of compression, forexample according to future MPEG standards or nay other standard.

Also, various elements may be implemented with a computer systemprogrammed with suitable computer programs that implement the invention,the programs at least including instructions for performing steps of amethod according to the invention when run on a computer system orenabling a general propose computer system to perform functions of acomputer system according to the invention. Such a computer program maybe provided on a data carrier, such as a CD-rom or diskette, stored withdata loadable in a memory of a computer system, the data representingthe computer program. A data carrier may further be a data connection,such as a telephone cable or a wireless connection transmitting signalsrepresenting a computer program according to the invention.

In the foregoing specification, the invention has been described withreference to specific examples of embodiments of the invention. Thespecifications and drawings are, accordingly, to be regarded in anillustrative rather than in a restrictive sense.

1. A method of encoding a digital video stream, the method comprisingthe steps of dividing frames of the video stream into blocks; selectingquantizer scale values for respective ones of the blocks, under controlof a complexity of a content of image information in the respective onesof the blocks, wherein an effect, of fluctuations of the complexity as afunction of time and/or position of the respective ones of the blocks,on temporal and or spatial fluctuations of the quantizer scale values isdampened; computing video data quantized according to the selectedquantizer scale values; generating an encoded video stream comprisingthe quantized video data.
 2. A method of encoding a digital video streamaccording to claim 1, wherein the selecting step comprises preselectingpreliminary values for the quantizer scale values for respective ones ofthe blocks, under control of the complexity; temporally and or spatiallylow-pass filtering the preliminary values to determine the quantizerscale values to be used in the computing step.
 3. A method of encoding adigital video stream according to claim 1, wherein the digital videostream is an encoded video stream comprising information representingquantized input values, and requantizing the quantized input valuesaccording to the selected quantizer scale values.
 4. A method ofencoding a digital video stream according to claim 1, the methodcomprising relating a succession of blocks from successive frames toeach other on the basis of estimated motion vectors, the selecting stepcomprising temporally low-pass filtering a succession of quantizationvalues that are provisionally selected for the successive blocksrespectively.
 5. A system for encoding a digital video stream, thesystem comprising a pre-processor arranged to compute image informationfor respective blocks of pixels in frames of the video stream; aquantizer arranged to quantize the computed image information, usingblock dependent quantizer scale values; an encoded video streamgenerator arranged to generate an encoded stream that encodes thequantized video data; a quantization scale selector arranged to selectthe quantizer scale values for respective ones of the blocks, undercontrol of a complexity of a content of image information in therespective ones of the blocks, wherein an effect, of fluctuations of thecomplexity as a function of time and/or position of the respective onesof the blocks, on temporal and or spatial fluctuations of the quantizerscale values is dampened.
 6. A system according to claim 5, the systemcomprising a preselector for preselecting preliminary values for thequantizer scale values for respective ones of the blocks, under controlof the complexity; a low pass filter unit for temporally and orspatially low-pass filtering the preliminary values to determine thequantizer scale values.
 7. A system according to claim 5, thepre-processor being arranged to compute the image information from anincoming encoded video stream with quantized input signal values, thequantizer requantizing the quantized input values.
 8. A system accordingto claim 6, the filter temporally low pass filtering preselectedquantization that are pre-selected for a succession of selected blocksin different frames, the selected blocks being related to each other bymotion vectors estimated for the frames.
 9. A computer program productcomprising a computer program with instructions for programming acomputer to execute the method according to claim 1.